Systems and methods for digital vaccine

ABSTRACT

We disclose a digital vaccine system which presents a user-driven avatar with tasks that test the avatar&#39;s physical fitness and food offerings at various stages. The avatar&#39;s appearance is responsive to the avatar&#39;s performance on the tasks and selection of the food offerings. The digital vaccine system uses deep learning systems to configure and update its parameters.

FIELD OF THE TECHNOLOGY DISCLOSED

The technology disclosed relates to artificial intelligence typecomputers and digital data processing systems and corresponding dataprocessing methods and products for emulation of intelligence (i.e.,knowledge based systems, reasoning systems, and knowledge acquisitionsystems); and including systems for reasoning with uncertainty (e.g.,fuzzy logic systems), adaptive systems, machine learning systems, andartificial neural networks. In particular, the technology disclosedrelates to using deep neural networks such as convolutional neuralnetworks (CNNs) and fully-connected neural networks (FCNNs) foranalyzing data.

BACKGROUND

The subject matter discussed in this section should not be assumed to beprior art merely as a result of its mention in this section. Similarly,a problem mentioned in this section or associated with the subjectmatter provided as background should not be assumed to have beenpreviously recognized in the prior art. The subject matter in thissection merely represents different approaches, which in and ofthemselves can also correspond to implementations of the claimedtechnology.

Machine Learning

In machine learning input variables are used to predict an outputvariable. The input variables are often called features and are denotedby X=(X₁, X₂, . . . , X_(k)), where each X_(i), i∈1, . . . , k is afeature. The output variable is often called the response or dependentvariable and is denoted by the variable Y_(i). The relationship betweenY and the corresponding X can be written in a general form:Y=ƒ(X)+∈

In the equation above, ƒ is a function of the features (X₁, X₂, . . . ,X_(k)) and ∈ is the random error term. The error term is independent ofX and has a mean value of zero.

In practice, the features X are available without having Y or knowingthe exact relation between X and Y. Since the error term has a meanvalue of zero, the goal is to estimate ƒ.Ŷ={circumflex over (ƒ)}=(X)

In the equation above, {circumflex over (ƒ)} is the estimate of ∈, whichis often considered a black box, meaning that only the relation betweenthe input and output of {circumflex over (ƒ)} is known, but the questionwhy it works remains unanswered.

The function {circumflex over (ƒ)} is found using learning. Supervisedlearning and unsupervised learning are two ways used in machine learningfor this task. In supervised learning, labeled data is used fortraining. By showing the inputs and the corresponding outputs (=labels),the function {circumflex over (ƒ)} is optimized such that itapproximates the output. In unsupervised learning, the goal is to find ahidden structure from unlabeled data. The algorithm has no measure ofaccuracy on the input data, which distinguishes it from supervisedlearning.

Neural Networks

The single layer perceptron (SLP) is the simplest model of a neuralnetwork. It comprises one input layer and one activation function. Theinputs are passed through the weighted graph. The function ƒ uses thesum of the inputs as argument and compares this with a threshold θ.

A neural network is a system of interconnected artificial neurons (e.g.,a₁, a₂, a₃) that exchange messages between each other. The illustratedneural network has three inputs, two neurons in the hidden layer and twoneurons in the output layer. The hidden layer has an activation functionƒ(●) and the output layer has an activation function g(●). Theconnections have numeric weights (e.g., w₁₁, w₂₁, w₁₂, w₃₁, w₂₂, w₃₂,v₁₁, v₂₂) that are tuned during the training process, so that a properlytrained network responds correctly when fed an image to recognize. Theinput layer processes the raw input, the hidden layer processes theoutput from the input layer based on the weights of the connectionsbetween the input layer and the hidden layer. The output layer takes theoutput from the hidden layer and processes it based on the weights ofthe connections between the hidden layer and the output layer. Thenetwork includes multiple layers of feature-detecting neurons. Eachlayer has many neurons that respond to different combinations of inputsfrom the previous layers. These layers are constructed so that the firstlayer detects a set of primitive patterns in the input image data, thesecond layer detects patterns of patterns and the third layer detectspatterns of those patterns.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to like partsthroughout the different views. Also, the drawings are not necessarilyto scale, with an emphasis instead generally being placed uponillustrating the principles of the technology disclosed. In thefollowing description, various implementations of the technologydisclosed are described with reference to the following drawings.

FIG. 1 illustrates one implementation of the technology disclosedoperating in a digital vaccine environment.

FIG. 2 shows one example of the avatar data.

FIG. 3 depicts one example of the user data.

FIG. 4 illustrates one implementation of the nutrition data generationsystem.

FIG. 5 shows one implementation of the data processing system.

FIG. 6 depicts one example of the environment interaction data.

FIG. 7 shows one implementation of the modification system.

FIGS. 8A and 8B illustrate a recurrent neural network.

FIG. 9 illustrates an example LSTM block.

FIG. 10 depicts one implementation of workings of a convolutional neuralnetwork.

FIG. 11 depicts a block diagram of training a convolutional neuralnetwork in accordance with one implementation of the technologydisclosed.

FIG. 12 shows one implementation of a ReLU non-linear layer inaccordance with one implementation of the technology disclosed.

FIG. 13 illustrates dilated convolutions.

FIG. 14 is one implementation of sub-sampling layers (average/maxpooling) in accordance with one implementation of the technologydisclosed.

FIG. 15 depicts one implementation of a two-layer convolution of theconvolution layers.

FIG. 16 illustrates one implementation of a training stage in which thedata processing system and the modification system are trained.

FIG. 17 shows different types of neural networks that can be used by thedata processing system and the modification system.

FIG. 18 depicts one implementation of a method for artificialintelligence-controlled neuro physiological-behavior state modulation tolower health risk score.

FIG. 19 depicts one implementation of a method for personalization ofprecision health risk mapping.

FIG. 20 is a simplified block diagram of a computer system that can beused to implement the technology disclosed.

DETAILED DESCRIPTION

The following discussion is presented to enable any person skilled inthe art to make and use the technology disclosed, and is provided in thecontext of a particular application and its requirements. Variousmodifications to the disclosed implementations will be readily apparentto those skilled in the art, and the general principles defined hereinmay be applied to other implementations and applications withoutdeparting from the spirit and scope of the technology disclosed. Thus,the technology disclosed is not intended to be limited to theimplementations shown, but is to be accorded the widest scope consistentwith the principles and features disclosed herein.

The following description and drawings are illustrative and are not tobe construed as limiting. Numerous specific details are described toprovide a thorough understanding of the disclosure. However, in certaininstances, well-known or conventional details are not described in orderto avoid obscuring the description. References to one or animplementation in the present disclosure can be, but not necessarilyare, references to the same implementation; and, such references mean atleast one of the implementations.

Reference in this specification to “one implementation” or “animplementation” means that a particular feature, structure, orcharacteristic described in connection with the implementation isincluded in at least one implementation of the disclosure. Theappearances of the phrase “in one implementation” in various places inthe specification are not necessarily all referring to the sameimplementation, nor are separate or alternative implementations mutuallyexclusive of other implementations. Moreover, various features aredescribed which may be exhibited by some implementations and not byothers. Similarly, various requirements are described which may berequirements for some implementations but not other implementations.

The terms used in this specification generally have their ordinarymeanings in the art, within the context of the disclosure, and in thespecific context where each term is used. Certain terms that are used todescribe the disclosure are discussed below, or elsewhere in thespecification, to provide additional guidance to the practitionerregarding the description of the disclosure. For convenience, certainterms may be highlighted, for example using italics and/or quotationmarks. The use of highlighting has no influence on the scope and meaningof a term; the scope and meaning of a term is the same, in the samecontext, whether or not it is highlighted. It will be appreciated thatthe same thing can be said in more than one way.

Consequently, alternative language and synonyms may be used for any oneor more of the terms discussed herein, nor is any special significanceto be placed upon whether or not a term is elaborated or discussedherein. Synonyms for certain terms are provided. A recital of one ormore synonyms does not exclude the use of other synonyms. The use ofexamples anywhere in this specification including examples of any termsdiscussed herein is illustrative only, and is not intended to furtherlimit the scope and meaning of the disclosure or of any exemplifiedterm. Likewise, the disclosure is not limited to various implementationsgiven in this specification.

Without intent to further limit the scope of the disclosure, examples ofinstruments, apparatus, methods and their related results according tothe implementations of the present disclosure are given below. Note thattitles or subtitles may be used in the examples for convenience of areader, which in no way should limit the scope of the disclosure. Unlessotherwise defined, all technical and scientific terms used herein havethe same meaning as commonly understood by one of ordinary skill in theart to which this disclosure pertains. In the case of conflict, thepresent document, including definitions will control.

Implementations of the present disclosure include systems, methods andapparatuses of seamless integration of augmented, alternate, virtual,and/or mixed realities with physical realities for enhancement of web,mobile and/or other digital experiences. Implementations of the presentdisclosure further include systems, methods and apparatuses tofacilitate physical and non-physical interaction/action/reactionsbetween alternate realities.

The disclosed digital vaccine platform enables and facilitatesauthoring, discovering, and/or interacting with virtual objects (VOBs).One example implementation includes a system and a platform that canfacilitate human interaction or engagement with virtual objects(hereinafter, ‘VOB,’ or ‘VOBs’) in a digital realm (e.g., an augmentedreality environment (AR), an alternate reality environment (AR), a mixedreality environment (MR) or a virtual reality environment (VR)). Thehuman interactions or engagements with VOBs in or via the disclosedenvironment can be integrated with and bring utility to everyday livesthrough integration, enhancement or optimization of our digitalactivities such as web browsing, digital (online, or mobile shopping)shopping, socializing (e.g., social networking, sharing of digitalcontent, maintaining photos, videos, other multimedia content), digitalcommunications (e.g., messaging, emails, SMS, mobile communicationchannels, etc.), business activities (e.g., document management,document procession), business processes (e.g., IT, HR, security, etc.),transportation, travel, etc.

The disclosed digital vaccine platform provides another dimension todigital activities through integration with the real world environmentand real world contexts to enhance utility, usability, relevancy,entertainment and/or vanity value through optimized contextual, social,spatial, temporal awareness and relevancy. In general, the virtualobjects depicted via the disclosed system and platform can becontextually (e.g., temporally, spatially, socially, user-specific,etc.) relevant and/or contextually aware. Specifically, the virtualobjects can have attributes that are associated with or relevant to realworld places, real world events, humans, real world entities, real worldthings, real world objects, real world concepts and/or times of thephysical world, and thus its deployment as an augmentation of a digitalexperience provides additional real life utility.

Note that in some instances, VOBs can be geographically, spatiallyand/or socially relevant and/or further possess real life utility. Inaccordance with implementations of the present disclosure, VOBs can beor appear to be random in appearance or representation with little to noreal world relation and have little to marginal utility in the realworld. It is possible that the same VOB can appear random or of littleuse to one human user while being relevant in one or more ways toanother user in the AR environment or platform.

The disclosed digital vaccine platform enables users to interact withVOBs and deployed environments using any device, including by way ofexample, computers, PDAs, phones, mobile phones, tablets, head mounteddevices, goggles, smart watches, monocles, smart lens, smart watches andother smart apparel (e.g., smart shoes, smart clothing), and any othersmart devices.

In one implementation, the disclosed digital vaccine platform isanalogous to, or operates in conjunction with the Web for the physicalworld. The host server can provide a browser, a hosted server, and asearch engine, for this new Web.

Implementations of the disclosed digital vaccine platform enablescontent (e.g., VOBs, third party applications, AR-enabled applications,or other objects) to be created and placed into layers (e.g., componentsof the virtual world, namespaces, virtual world components, digitalnamespaces, etc.) that overlay dietary information by anyone, andfocused around a layer that has the highest number of audience (e.g., apublic layer). The public layer can in some instances, be the maindiscovery mechanism and source for advertising venue for monetizing thedisclosed platform.

In one implementation, the disclosed digital vaccine platform includes avirtual world that exists in another dimension superimposed on thephysical world. Users can perceive, observe, access, engage with orotherwise interact with this virtual world via a user interface ofclient application.

One implementation of the disclosed digital vaccine platform includes aconsumer or client application component (e.g., as deployed on userdevices) which is able to provide dietary awareness to human users ofthe AR environment and platform. The client application can sense,detect or recognize virtual objects and/or other human users, actors,non-player characters or any other human or computer participants thatare within range of their physical location, and can enable the users toobserve, view, act, interact, react with respect to the VOBs.

Furthermore, implementations of the disclosed digital vaccine platformalso include an enterprise application (which can be desktop, mobile orbrowser based application). In this case, retailers, advertisers,merchants or third-party e-commerce platforms/sites/providers can accessthe disclosed platform through the enterprise application which enablesmanagement of paid advertising campaigns deployed via the platform.

Users can access the client application which connects to the hostplatform (e.g., as hosted by a host server). The client applicationenables users to sense and interact with virtual objects (“VOBs”) andother users (“Users”), actors, non-player characters, players, or otherparticipants of the platform. The VOBs can be marked or tagged (by QRcode, other bar codes, or image markers) for detection by the clientapplication.

The client devices can be any system and/or device, and/or anycombination of devices/systems that is able to establish a connectionwith another device, a server and/or other systems. Client devices eachtypically include a display and/or other output functionalities topresent information and data exchanged between among the devices and thehost server.

For example, the client devices can include mobile, hand held orportable devices or non-portable devices and can be any of, but notlimited to, a server desktop, a desktop computer, a computer cluster, orportable devices including, a notebook, a laptop computer, a handheldcomputer, a palmtop computer, a mobile phone, a cell phone, a smartphone, a PDA, a Blackberry device, a Treo, a handheld tablet (e.g. aniPad, a Galaxy, Xoom Tablet, etc.), a tablet PC, a thin-client, a handheld console, a hand held gaming device or console, an iPhone, awearable device, a head mounted device, a smart watch, a goggle, a smartglasses, a smart contact lens, and/or any other portable, mobile, handheld devices, etc. The input mechanism on client devices can includetouch screen keypad (including single touch, multi-touch, gesturesensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, atrack pad, motion detector (e.g., including 1-axis, 2-axis, 3-axisaccelerometer, etc.), a light sensor, capacitance sensor, resistancesensor, temperature sensor, proximity sensor, a piezoelectric device,device orientation detector (e.g., electronic compass, tilt sensor,rotation sensor, gyroscope, accelerometer), eye tracking, eye detection,pupil tracking/detection, or a combination of the above.

The client devices, application publisher/developer, its respectivenetworks of users, a third-party content provider, and/or promotionalcontent server, can be coupled to the network and/or multiple networks.In some implementations, the client devices and host server may bedirectly connected to one another. The alternate, augmented provided ordeveloped by the application publisher/developer can include anydigital, online, web-based and/or mobile based environments includingenterprise applications, entertainment platforms, gaming platforms,social networking platforms, e-commerce, exchanges, search platforms,browsing, discovery, messaging, chatting, and/or any other types ofactivities (e.g., network-enabled activities).

Motivation

Artificial Intelligence (AI) holds vast potential in reshaping the humancondition.

Entire industries are poised for being rescripted at their core. Much ofAI has been oriented towards replacing human tasks. Image recognition,Natural Language Processing, news feed/digital content curation andclick-through optimization, represent narrow use cases of AI inenhancing advertising-driven business models of digital contentplatforms. AI is now making headway, as human decision making can bereplaced, by better informed algorithms at greater speed. Driverlesscars, robotics, drones, transportation, manufacturing, etc representanother exciting facet of application of AI, especially as it canminimize human error, within the context of improving efficiency withinpre-existing industries. Neural networks represent one of the mostpowerful elements of AI, as outcomes can begin to mirror biologicalintelligence that can learn to adapt based on experience, powered bydata. Such advances could conceivably culminate in human vs AI conflict,especially in cases when AI-enabled precision outcomes, become pawns ofrogue intent. While most of the focus and directionality for AI havebeen towards automation of existing processes through data mining andmachine learning, the AI revolution holds promise for inverting theparadigm, with respect to several planetary-scale crises. There is anexciting role for AI, to empower human beings in unprecedented ways.Human intelligence and consciousness can be elevated through a morecapable and well-intentioned greater force. The creation of such AIwould call for a tremendous sense of purpose and deeper extents ofethical innovation by the designers of such AI-enabled augmentation.

This invention is meant to fulfill a vision to eradicate the preventablenutrition and lifestyle related global burden of diabetes,cardiovascular disease, hypertension, kidney disease, liver disease,cognitive disease and cancer, as well as find relevance in reducingsymptoms of illness, through our platform of DV+AI.

The past decade has been dedicated towards a high standard ofself-funded science and innovation, bound by ethical review and thescientific process of evidence-based iterative inquiry, towards amission to develop AI that can empower humans for generations to come.The fundamental realization has been that humans are already attackingour own, as unbridled profit motive turns a blind eye to human-planetarycost. The fundamental global trend that must be opposed, is that thereis vast profit captured by few, from the treatment, diagnosis, andperpetuation of human sickness globally. Enormous profit is capturedthrough capitalism fueled efficiency, increasingly AI-poweredalgorithmic assault, upon human health at the individual and societallevel. The long list of symptomatic evidence of this tragic realityranges from the proliferation of conglomerate fueled processed food,which feeds the other end of the spectrum—the colossal force of modernmedicine and big pharmaceutical companies. Both these domineeringindustries profit from undermining human potential.

It is an open secret that modern medicine has reached vulgar levels ofethical and moral conflict, as doctors in corporate PE controlledhospitals, make commissions and are manipulated to meet incentivized“sales quotas” on one hand, while they dishonorably prescribe moreinvasively debilitating medications that cause more harm than thediseases they claim to treat. “Do no harm” is simply not reconcilablewhen there is increasingly more money at stake from over-diagnosing,over-treating, creating more long-lasting dependency, which isultimately killing people. The Insulin crisis, the opioid crisis, thecholesterol sham, the chemotherapy scam, thyroid treatment, increasingC-sections, PE fueled price gouging, are just the proverbial tip of theiceberg, representing the extent and scale of injustice againsthumanity. It is way overdue that we restore the human condition tolevels of natural equilibrium embodied in ancient cultures andtraditions, which were designed at a time when wealth did not conflictwith human health. One such familiar source of inspiration for our teamis the Ayurveda, which is built upon a foundational commitment to“Swasthasya swasthya rakshanam. Aturasya roga nivaranam” (To protecthealth. To prevent disease—Charaka Samhita 500 B.C). One can contrastsuch a gentle biologically aligned philosophy to our modern-day reality,where dubious treatments backed by ulterior profit motives, becomeveiled behind the Hippocratic Oath (Do no harm) to capitalize onbiologically violent, protocol-driven treatments, which are validatedvia an irreproducible reductionist lens of evidence.

Digital vaccine (DV) plus (+) artificial intelligence (AI) exists torescript this global trend, at scale, with a solemn aim to empowerhumanity by protecting our health and enhancing human potential byreducing health risk. This synthesis of technology is designed to attainthis goal, while living up to sublime standards that can reorientcapitalism, in favor of protecting health, rather than treatingsickness. Our methods of DV are built upon fundamental neurosciencebreakthroughs in Neuromodulation and neurostimulation, at themechanistic and physiological level, through non-invasive technology.This allows development of deep technology and fundamental know-how.

The investment thesis in favor of DV+AI is quite simple. The roughestimate of the aggregate of the global market cap of pharmaceuticalcompanies, hospital systems and healthcare delivery spend, amounts tomore than $15 Trillion. The economic burden of global healthcare annualspend is a staggering $20+ Trillion. The vast burden of disease hasshifted, while continuing to shift, towards profit fueled preventableNon Communicable Diseases (NCDs). The even more macabre forecast is thatthese numbers are projected to grow with unforeseen acceleration.Through highly scalable DV+AI, a small fraction of the economic valuecreated can be captured, as DV+AI will disintermediate the currentcapitalistic machinery that profits from sickness. Given the earlystages of the field being defined and the commercialization tractionthrough a current SaaS-like monthly subscription model (distributing toeach student via mobile devices) a mandatory health educationco-curriculum and co-scholastic program is being shaped, as a result ofdistribution through qualified distinguished school partners of the DVproject. DV+AI is already getting to the market by filling a void forclinically proven science-based nutrition-health education curriculum.Given that DV+AI is a subset of DTx, this is particularly significantgiven the current state of confusion about viable go to market modelsamong contemporary adjacent DTx companies, many of whom have raisedseveral 100s of millions of dollars, to tackle much smaller scope ofdisease through non-scalable technology. As continuing focus onpopulation health grows DV+AI will attain levels of evidence thatbecomes irrefutable from a reimbursement perspective. Based on our timetested partnerships with world leading researchers at academic researchinstitutions, like Carnegie Mellon University, Stanford School ofMedicine, Stanford Law School, Johns Hopkins University Bloomberg Schoolof Public Health, Pittsburgh Children's Hospital of UPMC, OxfordUniversity Nuffield Department of Medicine, Baylor College of Medicine,National University of Singapore and the like, we aim to define thecategory and calibration standards of DV+AI. We are already preparingfor randomized controlled clinical trials, through our pre-existing andexpanding network of renowned life science researchers, with a clearscientific aim to measure outcomes of DV+AI on cholesterol (HDL/LDL),blood glucose, A1c, Ketones, Glycoprotein Acetyls, Amino acids, Bodymass Index BMI, Cognitive development markers. DV+AI is thereforesquarely in the turf of pharmaceutical and biotech companies. DV+AIpresents the world a non-invasive, relatively risk-free and rigorousscience-based alternative, to invasive medications and treatments, witha foundation of science will set a high bar for future competitors, bybuilding a moat based on medical-grade published longitudinal outcomes.Through partnerships, we will continue to push to set a global standardfor a compulsory DV for every child in our world, backed byrecommendations and policy frameworks from ministries ofhealth/education. DV+AI is also on the cusp of gaining endorsement fromorganizations such as World Health Organization (WHO)/United NationaChildren's Fund (UNICEF)/United Nations Development Program (UNDP). Thisfocus on science will keep go-to-market marketing costs low whileleveraging the groundbreaking science with world-renowned academicpartners. This unique synthesis of highly scalable software basedinnovation, which can be protected through an IP estate, will ensuresoftware levels of profitability.

An audacious goal is to create a company with a multi-century horizonthat surpasses market value in excess of $1 Trillion, driven byuncompromising mission to garner the requisite influence and resources,to rescript the future of human potential. This mission is alwaysrelevant because of a belief that good health will always be crucial fortrue happiness and therefore, will remain the greatest wealth to bepassed on to future generations.

Digital Vaccine Environment

We describe a system and various implementations for providing a digitalvaccine solution. The system and processes are described with referenceto FIG. 1. Because FIG. 1 is an architectural diagram, certain detailsare intentionally omitted to improve the clarity of the description. Thediscussion of FIG. 1 is organized as follows. First, the elements of thefigure are described, followed by their interconnections. Then, the useof the elements is described in greater detail.

FIG. 1 illustrates one implementation of the technology disclosedoperating in a digital vaccine environment 100. User 102 uses devicessuch as smartphones, tablets, laptops, and personal computers (PCs) tointerface with the digital vaccine environment 100. The digital vaccineenvironment 100 is responsive to user input 112 provided by the user102.

The digital vaccine environment 100 can be run by a game processor likephysics engine 140, which implements the digital vaccine environment 100in a gamified context centered at an avatar 126. The physics engine 140can be UNITY 3D™ or HAVOK™. The physics engine 140 can be configuredwith logic that specifies the narrative, stages, tasks, animations, andsimulations of the digital vaccine environment 100 that the avatar 126goes through and interacts with, including rules that govern how theavatar 126 is modified as it operates within the digital vaccineenvironment 100 based on the user input 112.

The digital vaccine environment 100 further comprises an avatarappearance engine 106, a virtual input generation sub-system 136, aperformance database 114, a tasks database 118, a food offeringsdatabase 128, and an avatar data database 138.

The modules of the digital vaccine environment 100 can be implemented inhardware or software, and need not be divided up in precisely the sameblocks as shown in FIG. 1. Some of the modules can also be implementedon different processors or computers, or spread among a number ofdifferent processors or computers. In addition, it will be appreciatedthat some of the modules can be combined, operated in parallel or in adifferent sequence than that shown in FIG. 1 without affecting thefunctions achieved. Also as used herein, the term “module” can include“sub-modules,” which themselves can be considered to constitute modules.The blocks in the digital vaccine environment 100, designated asmodules, can also be thought of as flowchart steps in a method. A modulealso need not necessarily have all its code disposed contiguously inmemory; some parts of the code can be separated from other parts of thecode with code from other modules or other functions disposed inbetween.

The interconnections of the elements of the digital vaccine environment100 are now described. The actual communication path can bepoint-to-point over public and/or private networks. Some items might bedelivered indirectly, e.g., via an application store (not shown). Thecommunications can occur over a variety of networks, e.g., privatenetworks, VPN, MPLS circuit, or Internet, and can use appropriateapplication programming interfaces (APIs) and data interchange formats,e.g., Representational State Transfer (REST), JavaScript Object Notation(JSON), Extensible Markup Language (XML), Simple Object Access Protocol(SOAP), Java Message Service (JMS), and/or Java Platform Module System.All of the communications can be encrypted. The communication isgenerally over a network such as the LAN (local area network), WAN (widearea network), telephone network (Public Switched Telephone Network(PSTN), Session Initiation Protocol (SIP), wireless network,point-to-point network, star network, token ring network, hub network,Internet, inclusive of the mobile Internet, via protocols such as EDGE,3G, 4G LTE, Wi-Fi, and WiMAX. Additionally, a variety of authorizationand authentication techniques, such as username/password, OpenAuthorization (OAuth), Kerberos, SecureID, digital certificates, voicerecognition, fingerprint scan, facial recognition, biometric scanc andmore, can be used to secure the communications.

The digital vaccine environment 100 can be accessed via an applicationprogramming interface (API). An API refers to a packaged collection ofcode libraries, routines, protocols methods, and fields that belong to aset of classes, including its interface types. The API defines the waythat developers and programmers can use the classes for their ownsoftware development, just by importing the relevant classes and writingstatements that instantiate the classes and call their methods andfields. An API is a source code-based application intended to be used asan interface by software components to communicate with each other. AnAPI can include applications for routines, data structures, objectclasses, and variables. Basically, an API provides an interface fordevelopers and programmers to access the underlying data, platformcapabilities, and features of cloud-based services. Implementations ofthe technology disclosed use different types of APIs, including webservice APIs such as HTTP or HTTPs based APIs like SOAP, WSDL, Bulk,XML-RPC and JSON-RPC and REST APIs (e.g., FLICKR™, GOOGLE STATIC MAPS™,GOOGLE GEOLOCATION™), web socket APIs, library-based APIs likeJavaScript and TWAIN (e.g., GOOGLE MAPS™ JavaScript API, DROPBOX™JavaScript Data store API, TWILIO™ APIs, Oracle Call Interface (OCI)),class-based APIs like Java API and Android API (e.g., GOOGLE MAPS™Android API, MSDN Class Library for .NET Framework, TWILIO™ APIs forJava and C #), OS functions and routines like access to file system andaccess to user interface, object remoting APIs like CORBA and .NETRemoting, and hardware APIs like video acceleration, hard disk drives,and PCI buses. Other examples of APIs used by the technology disclosedinclude AMAZON EC2 API™, BOX CONTENT API™, BOX EVENTS API™, MICROSOFTGRAPH™, DROPBOX API™, DROPBOX API v2™, DROPBOX CORE API™, DROPBOX COREAPI v2™, FACEBOOK GRAPH API™, FOURSQUARE API™, GEONAMES API™, FORCE.COMAPI™, FORCE.COM METADATA API™, APEX API™, VISUALFORCE API™, FORCE.COMENTERPRISE WSDL™, SALESFORCE.COM STREAMING API™, SALESFORCE.COM TOOLINGAPI™, GOOGLE DRIVE API™, DRIVE REST API™, ACCUWEATHER API™, andaggregated-single API like CLOUDRAIL™ API.

Having introduced the elements of FIG. 1 and their interconnections,elements of the figure are now described in greater detail.

The digital vaccine environment 100 presents the user-driven avatar 126with (i) tasks 118 that test the avatar's physical fitness and (ii) foodofferings 128 at various stages 108 of the play (game). The appearanceof the avatar 126 is responsive to the avatar's performance 114 on thetasks 118 and selection of the food offerings 128. For example, theavatar 126 is offered increasingly difficult physical exertionchallenges as it progresses through the various stages 108 of the play(game). As the avatar 126 successfully performs the physical tasks, itsappearance becomes healthier and fitter (e.g., muscular, thinner). Inanother example, the avatar 126 is offered a variety of food types withdifferent nutritional values and calorie counts. Based on the foodconsumed, the appearance of the avatar 126 changes (e.g., less muscular,bulkier). In one implementation, the changes in the appearance of theavatar 126 are implemented by the avatar appearance engine 106. Also,the movement and activity of the avatar 126 is controlled by the userinput 112.

An input generation system has a virtual input generation sub-system 136and an actual input generation sub-system 304. The virtual inputgeneration sub-system 136 monitors the avatar's progression through thedigital vaccine environment 100 and produces avatar data 138.

FIG. 2 illustrates one example of the avatar data 138. In oneimplementation, the avatar data 138 includes (i) avatar food preferencedata 202, (ii) avatar calorie data 212, (iii) avatar insulin data 222,(iv) avatar glucose data 232, (v) avatar A1C data 242, and (vi) avatarketone data 252 (vii) avatar cholesterol (HDL/LDL) data (viii) avataramino acids data (ix) avatar glycoprotein acetyls data (x) avatar gutmicrobiome data.

The avatar food preference data 202 further comprises time stampedvirtual food presented to the avatar in the digital vaccine environment,and time stamped virtual food selected by the avatar in the digitalvaccine environment. The avatar calorie data 212 further comprises totalcalorie level of the avatar, calories expanded by the avatar as a resultof performing the tasks, and net calorie level of the avatar. The avatarinsulin data 222 further comprises virtual insulin dose counter, andvirtual insulin units. The avatar glucose data 232 further comprisesvirtual net blood glucose. The avatar A1C data 242 further comprisesvirtual A1C results. The avatar ketone data 252 further comprisesvirtual ketone level.

FIG. 3 illustrates one example of the user data 306. The actual inputgeneration sub-system 304 accesses a user information database 302 andproduces the user data 306. In one implementation, the user data 306includes (i) user food preference data 308, (ii) user calorie data 310,(iii) user glycemic data 312, (iv) user insulin data 314, (v) userglucose data 316, (vi) user A1C data 318, and (vii) user ketone data 320(vii) user cholesterol (HDL/LDL) data (viii) user amino acids data (ix)user glycoprotein acetyls data (x) user gut microbiome data (FIG. 3 mustbe modified to include these indicators).

In some implementations, the actual input generation sub-system 304 is aweb crawler that collects the user data 306 from online biographicsources such as social media sites, clinician input or Electronic HealthRecords.

The user food preference data 308 further comprises time stamped actualfood presented to the user in real world, and time stamped actual foodconsumed by the user in the real world. The user calorie data 310further comprises actual calories consumed by the user in the realworld, and calories expanded by the user in the real world. The userglycemic data 312 further comprises glycemic index, and glycemic load.The user insulin data 314 further comprises actual insulin dose counter,and actual insulin units. The user glucose data 316 further comprisesactual net blood glucose. The user A1C data 318 further comprises actualA1C results. The user ketone data 320 further comprises actual ketonelevel. The user HDL/LDL data further comprises the actual cholesteroldata. The user amino acids data further comprises actual amino acidlevel. The user glycoprotein acetyls data further comprises the actualglucoprotein acetyls level. The user gut microbiome data furthercomprises the actual gut microbiome data.

FIG. 4 illustrates one implementation of the nutrition data generationsystem 424. The nutrition data generation system 424 processes (i) foodlogs 412, (ii) user conversation files 422, (iii) user images 432,and/or (iv) food images 442 and produces nutrition data 426.

The nutrition data generation system 424 can use deep neural networks.Deep neural networks are a type of artificial neural networks that usemultiple nonlinear and complex transforming layers to successively modelhigh-level features. Deep neural networks provide feedback viabackpropagation which carries the difference between observed andpredicted output to adjust parameters.

Deep neural networks are a family of parametric, non-linear andhierarchical learning functions. Given a dataset D, deep neural networksneed to find the optimal parameters θ that minimize some loss function.These models are called networks because they are a collection offunctions that can be represented as an acyclic graph. The acyclic graphis divided into layers, and each layer represents a computation of theform:h ₁ =f ₁(W ₁ ·x+b ₁)where x is the multidimensional input of the model that is mapped to thehidden unit h₁ using weights W₁∈θ and biases b₁∈θ. The function f1(⋅) iscalled an activation function. The output of one layer can be used asinput for another layer.h ₂ =f ₂(W ₂ ·f ₁(W ₁ ·x+b ₁)+b ₂)

Hence the hierarchical aspect of neural networks. The field of deeplearning focuses on neural networks with a large number of these layersbecause they are capable of approximating more complex functions.

In one implementation, the nutrition data generation system 424 is arecurrent neural network that processes the user conversation files 422and produces the nutrition data 426. Recurrent neural networks (RNN) arepart of the family of deep neural networks and are able to processsequential data. To understand the information that is incorporated in asequence, an RNN needs memory to know the context of the data.Information about the past is passed through the network using hiddenstates. Therefore a single computational unit can be dependent on itsprevious states. The idea of using RNN's is to get a natural way of thepersistence of memory. The cycles allow the RNN's to get thispersistence behavior. FIG. 8A illustrates a schematic representation ofan RNN, where g is a part of a neural network and should not be confusedwith the activation function.

The unfolded network in FIG. 8B clarifies how an RNN works. It can beseen as a neural network composed of smaller neural networks in whichinformation is passed in an ordered way. The unfolded network shows thatif t represents the time, causality relations hold in these type ofnetworks. This makes the RNN interesting for studying time series. Inthe study of time series on a daily basis, seasonal features need to bedetected. The period of a season can become relatively large.Theoretically, it should be possible to learn any relation between thepast with the current time, since the information is passed through eachblock. However, learning long term dependencies for RNN's using gradientdescent algorithms is difficult.

The Long Short Term Memory (LSTM) recurrent neural network is designedto be able to learn these long-term relations without overlooking theshort-term dependencies. FIG. 9 illustrates a block of an LSTM. In FIG.9, × and + are point-wise operators, and σ and tan h are activationfunctions. Two joining arrows make a concatenate operation. Twosplitting arrows make a copy operation. The LSTM block is repeated inthe same way as the RNN. The LSTM block consists of two lines passinghorizontally, the y-value which corresponds to the output of a block andthe C-value which corresponds to the cell state. The horizontal lineshave inputs C_(t−1) 906 and y_(t−1) 908 from the preceding blocks, andoutputs C_(t) 910 and y_(t) 912 to the succeeding blocks. Vertically,for each block, there is an input x and an output y.

Starting with the input x_(t) 914, the signal is concatenated withy_(t−1) 908 to obtain [y_(t−1), x_(t)]. Following the first arrowpointing downwards, the values are passed through a sigmoid function σ916. The output f_(t) 918 of the sigmoid function σ 916 function isdefined as:f _(t)=σ(W _(f)·[y _(t−1) ,x _(t)]+b _(f))

The function above is called the forget gate, since the output, a valuein (0, 1), decides whether the preceding cell state is remembered orforgotten using the point-wise product operator.

Following the second arrow pointing downwards, the signal [y_(t−1),x_(t)] arrives at another sigmoid function σ 920 which is called theinput gate. The output i_(t) 922 decides which values are used for theupdate. The output i_(t) 922 is:i _(t)=σ(W _(i)·[y _(t−1) ,x _(t)]+b _(i))

The third arrow pointing downwards generates new candidate values C_(nt)924 for the cell state by using the tan h function 926. By taking thecross product with the input gate, the update for the cell state can bedetermined using:C _(nt)=tan h(W _(C)·[y _(t−1) ,x _(t)]+b _(C))C _(t) =f _(t) *C _(t−1) +i _(t) *C _(nt)

The new cell state is a combination of the old cell state and the newcandidate in which the forget gate and the input gate gradual decidewhether to use the old cell state and new input respectively.

The output gate o_(t) 930 transforms the signal [y_(t−1), x_(t)] asdefined by:o _(t)=σ(W _(o)[y _(t−1) ,x _(t)]+b _(o))

By taking the product of the tan h of the updated cell state Ct 910 andthe output gate o_(t) 930, the new output y_(t) 912 is defined as:y _(t) =o _(t)*tan h(C _(t))

The main components of the LSTM are the cell state and the output. Thenew cell state is defined by the forget gate and input gate. The newoutput is defined by the output gate and the new cell state. By adding nof these blocks, the size of the vectors passing through the blocks isgrowing linearly.

In other implementations, the nutrition data generation system 424 canbe an XGBoosted tree or a decision tree.

XGBoost stands for eXtreme Gradient Boosting, and it is a distributedimplementation of gradient boosting with emphasis on efficiency,flexibility, and portability. It provides parallel tree boosting and isfaster when compared with other gradient boosting implementation.

A decision tree is a model that begins with a single non-leaf node thatbranches into different outcomes. Then the outcomes lead to moreadditional nodes. Each non-leaf node represents the test on oneparticular feature, each branch represents the outcome of this feature,and each leaf node stores a classification. Once the split for eachfeature is done, the one with the minimum loss is viewed as the bestsplit criteria and set it as a rule for that node. The splitting processkeeps going until the termination condition is met.

Boosting technique holds the principle that a combination of weakclassifiers can create a single strong classifier. Weak classifiers areclassifiers that tend to perform insufficiently when applied inisolation but well when combined with other weak classifiers trained onthe same dataset. For the boosting methods, the additive training methodis applied in each step, during which a week classifier is added to themodel. In XGBoost, the weak classifier is the new decision tree.Equations below show this hallmark:F ₀=0F _(t)(x)=F _(t−1)(x)+h(x)where h(x) is the new decision tree after F_(t−1)(x) and F_(t)(x) is thenew model after t−1 steps. The objective of the XGBoost model is to findthe tree F_(t)(x) that minimizes the following equation at the t^(th)step:Obj(F _(t))=L(F _(t−1) +F _(t))+Ω(F _(t)).L is the loss function that decides the predictive power, and Ω is theregularization function controlling the overfitting.

In another implementation, the nutrition data generation system 424 is aconvolutional neural network that processes the user images 432 and/orthe food images 442 and produces the nutrition data 426.

A convolutional neural network is a special type of neural network. Thefundamental difference between a densely connected layer and aconvolution layer is this: Dense layers learn global patterns in theirinput feature space, whereas convolution layers learn local patterns: inthe case of images, patterns found in small 2D windows of the inputs.This key characteristic gives convolutional neural networks twointeresting properties: (1) the patterns they learn are translationinvariant and (2) they can learn spatial hierarchies of patterns.

Regarding the first, after learning a certain pattern in the lower-rightcorner of a picture, a convolution layer can recognize it anywhere: forexample, in the upper-left corner. A densely connected network wouldhave to learn the pattern anew if it appeared at a new location. Thismakes convolutional neural networks data efficient because they needfewer training samples to learn representations they have generalizationpower.

Regarding the second, a first convolution layer can learn small localpatterns such as edges, a second convolution layer will learn largerpatterns made of the features of the first layers, and so on. Thisallows convolutional neural networks to efficiently learn increasinglycomplex and abstract visual concepts.

A convolutional neural network learns highly non-linear mappings byinterconnecting layers of artificial neurons arranged in many differentlayers with activation functions that make the layers dependent. Itincludes one or more convolutional layers, interspersed with one or moresub-sampling layers and non-linear layers, which are typically followedby one or more fully connected layers. Each element of the convolutionalneural network receives inputs from a set of features in the previouslayer. The convolutional neural network learns concurrently because theneurons in the same feature map have identical weights. These localshared weights reduce the complexity of the network such that whenmulti-dimensional input data enters the network, the convolutionalneural network avoids the complexity of data reconstruction in featureextraction and regression or classification process.

Convolutions operate over 3D tensors, called feature maps, with twospatial axes (height and width) as well as a depth axis (also called thechannels axis). For an RGB image, the dimension of the depth axis is 3,because the image has three color channels; red, green, and blue. For ablack-and-white picture, the depth is 1 (levels of gray). Theconvolution operation extracts patches from its input feature map andapplies the same transformation to all of these patches, producing anoutput feature map. This output feature map is still a 3D tensor: it hasa width and a height. Its depth can be arbitrary, because the outputdepth is a parameter of the layer, and the different channels in thatdepth axis no longer stand for specific colors as in RGB input; rather,they stand for filters. Filters encode specific aspects of the inputdata: at a height level, a single filter could encode the concept“presence of a face in the input,” for instance.

For example, the first convolution layer takes a feature map of size(28, 28, 1) and outputs a feature map of size (26, 26, 32): it computes32 filters over its input. Each of these 32 output channels contains a26×26 grid of values, which is a response map of the filter over theinput, indicating the response of that filter pattern at differentlocations in the input. That is what the term feature map means: everydimension in the depth axis is a feature (or filter), and the 2D tensoroutput [:, :, n] is the 2D spatial map of the response of this filterover the input.

Convolutions are defined by two key parameters: (1) size of the patchesextracted from the inputs—these are typically 1×1, 3×3 or 5×5 and (2)depth of the output feature map—the number of filters computed by theconvolution. Often these start with a depth of 32, continue to a depthof 64, and terminate with a depth of 128 or 256.

A convolution works by sliding these windows of size 3×3 or 5×5 over the3D input feature map, stopping at every location, and extracting the 3Dpatch of surrounding features (shape (window_height, window_width,input_depth)). Each such 3D patch is ten transformed (via a tensorproduct with the same learned weight matrix, called the convolutionkernel) into a 1D vector of shape (output_depth). All of these vectorsare then spatially reassembled into a 3D output map of shape (height,width, output_depth). Every spatial location in the output feature mapcorresponds to the same location in the input feature map (for example,the lower-right corner of the output contains information about thelower-right corner of the input). For instance, with 3×3 windows, thevector output [i, j, :] comes from the 3D patch input [i−1:i+1, j−1:J+1,:]. The full process is detailed in FIG. 10.

The convolutional neural network comprises convolution layers whichperform the convolution operation between the input values andconvolution filters (matrix of weights) that are learned over manygradient update iterations during the training. Let (m, n) be the filtersize and W be the matrix of weights, then a convolution layer performs aconvolution of the W with the input X by calculating the dot productW·x+b, where x is an instance of X and b is the bias. The step size bywhich the convolution filters slide across the input is called thestride, and the filter area (m×n) is called the receptive field. A sameconvolution filter is applied across different positions of the input,which reduces the number of weights learned. It also allows locationinvariant learning, i.e., if an important pattern exists in the input,the convolution filters learn it no matter where it is in the sequence.

Training a Convolutional Neural Network

FIG. 11 depicts a block diagram of training a convolutional neuralnetwork in accordance with one implementation of the technologydisclosed. The convolutional neural network is adjusted or trained sothat the input data leads to a specific output estimate. Theconvolutional neural network is adjusted using back propagation based ona comparison of the output estimate and the ground truth until theoutput estimate progressively matches or approaches the ground truth.

The convolutional neural network is trained by adjusting the weightsbetween the neurons based on the difference between the ground truth andthe actual output. This is mathematically described as:Δ

_(i) =x _(i)δwhere δ=(ground truth)−(actual output)

In one implementation, the training rule is defined as:

_(nm)←

_(nm)+α(t _(m)−φ_(m))a _(n)

In the equation above: the arrow indicates an update of the value; t_(m)is the target value of neuron m; φ_(m) is the computed current output ofneuron m; a_(n) is input n; and α is the learning rate.

The intermediary step in the training includes generating a featurevector from the input data using the convolution layers. The gradientwith respect to the weights in each layer, starting at the output, iscalculated. This is referred to as the backward pass, or goingbackwards. The weights in the network are updated using a combination ofthe negative gradient and previous weights.

In one implementation, the convolutional neural network uses astochastic gradient update algorithm (such as ADAM) that performsbackward propagation of errors by means of gradient descent. One exampleof a sigmoid function based back propagation algorithm is describedbelow:

$\varphi = {{f(h)} = \frac{1}{1 + e^{- h}}}$

In the sigmoid function above, h is the weighted sum computed by aneuron. The sigmoid function has the following derivative:

$\frac{\partial\varphi}{\partial h} = {\varphi\left( {1 - \varphi} \right)}$

The algorithm includes computing the activation of all neurons in thenetwork, yielding an output for the forward pass. The activation ofneuron m in the hidden layers is described as:

$\varphi_{m} = \frac{1}{1 + e^{- {hm}}}$$h_{m} = {\sum\limits_{n = 1}^{N}{a_{n}w_{n\; m}}}$

This is done for all the hidden layers to get the activation describedas:

$\varphi_{k} = \frac{1}{1 + e^{hk}}$$h_{k} = {\sum\limits_{m = 1}^{M}\varphi_{m^{v}mk}}$

Then, the error and the correct weights are calculated per layer. Theerror at the output is computed as:δ_(ok)=(t _(k)−φ_(k))φ_(k)(1−φ_(k))

The error in the hidden layers is calculated as:

$\delta_{hm} = {{\varphi_{m}\left( {1 - \varphi_{m}} \right)}{\sum\limits_{k = 1}^{K}{v_{mk}\delta_{ok}}}}$

The weights of the output layer are updated as:

mk←

mk+αδokφm

The weights of the hidden layers are updated using the learning rate αas:

nm←

nm+αδhman

In one implementation, the convolutional neural network uses a gradientdescent optimization to compute the error across all the layers. In suchan optimization, for an input feature vector x and the predicted outputŷ, the loss function is defined as l for the cost of predicting ŷ whenthe target is y, i.e. l (ŷ, y). The predicted output ŷ is transformedfrom the input feature vector x using function ƒ. Function ƒ isparameterized by the weights of convolutional neural network, i.e.ŷ=ƒ_(w)(x). The loss function is described as l (ŷ, y)=l (ƒ_(w)(x), y),or Q (z, w)=l (ƒ_(w)(x), y) where z is an input and output data pair (x,y). The gradient descent optimization is performed by updating theweights according to:

$v_{t + 1} = {{\mu\; v_{t}} - {\alpha\;\frac{1}{n}{\sum\limits_{i = 1}^{N}{{\nabla w_{t}}{Q\left( {z_{t},w_{t}} \right)}}}}}$w_(t + 1) = w_(t) + v_(t + 1)

In the equations above, α is the learning rate. Also, the loss iscomputed as the average over a set of n data pairs. The computation isterminated when the learning rate α is small enough upon linearconvergence. In other implementations, the gradient is calculated usingonly selected data pairs fed to a Nesterov's accelerated gradient and anadaptive gradient to inject computation efficiency.

In one implementation, the convolutional neural network uses astochastic gradient descent (SGD) to calculate the cost function. A SGDapproximates the gradient with respect to the weights in the lossfunction by computing it from only one, randomized, data pair, Z_(t),described as:

_(t+1) =μ

−α∇wQ(z _(t) ,w _(t))

_(t+1)=

_(t)+

_(t+1)

In the equations above: α is the learning rate; μ is the momentum; and tis the current weight state before updating. The convergence speed ofSGD is approximately O(1/t) when the learning rate α are reduced bothfast and slow enough. In other implementations, the convolutional neuralnetwork uses different loss functions such as Euclidean loss and softmaxloss. In a further implementation, an Adam stochastic optimizer is usedby the convolutional neural network.

Convolution Layers

The convolution layers of the convolutional neural network serve asfeature extractors. Convolution layers act as adaptive featureextractors capable of learning and decomposing the input data intohierarchical features. In one implementation, the convolution layerstake two images as input and produce a third image as output. In such animplementation, convolution operates on two images in two-dimension(2D), with one image being the input image and the other image, calledthe “kernel”, applied as a filter on the input image, producing anoutput image. Thus, for an input vector ƒ of length n and a kernel g oflength m, the convolution ƒ*g of ƒ and g is defined as:

${\left( {f*g} \right)(i)} = {\sum\limits_{j = 1}^{m}{{g(j)} \cdot {f\left( {i - j + {m/2}} \right)}}}$

The convolution operation includes sliding the kernel over the inputimage. For each position of the kernel, the overlapping values of thekernel and the input image are multiplied and the results are added. Thesum of products is the value of the output image at the point in theinput image where the kernel is centered. The resulting differentoutputs from many kernels are called feature maps.

Once the convolutional layers are trained, they are applied to performrecognition tasks on new inference data. Since the convolutional layerslearn from the training data, they avoid explicit feature extraction andimplicitly learn from the training data. Convolution layers useconvolution filter kernel weights, which are determined and updated aspart of the training process. The convolution layers extract differentfeatures of the input, which are combined at higher layers. Theconvolutional neural network uses a various number of convolutionlayers, each with different convolving parameters such as kernel size,strides, padding, number of feature maps and weights.

Non-Linear Layers

FIG. 12 shows one implementation of non-linear layers in accordance withone implementation of the technology disclosed. Non-linear layers usedifferent non-linear trigger functions to signal distinct identificationof likely features on each hidden layer. Non-linear layers use a varietyof specific functions to implement the non-linear triggering, includingthe rectified linear units (ReLUs), hyperbolic tangent, absolute ofhyperbolic tangent, sigmoid and continuous trigger (non-linear)functions. In one implementation, a ReLU activation implements thefunction y=max(x, 0) and keeps the input and output sizes of a layer thesame. The advantage of using ReLU is that the convolutional neuralnetwork is trained many times faster. ReLU is a non-continuous,non-saturating activation function that is linear with respect to theinput if the input values are larger than zero and zero otherwise.Mathematically, a ReLU activation function is described as:

φ(h) = max (h, 0) ${\varphi(h)} = \left\{ \begin{matrix}{{h\mspace{14mu}{if}\mspace{14mu} h} > 0} \\{{0\mspace{14mu}{if}\mspace{14mu} h} \leq 0}\end{matrix} \right.$

In other implementations, the convolutional neural network uses a powerunit activation function, which is a continuous, non-saturating functiondescribed by:φ(h)=(a+bh)^(c)

In the equation above, a, b and c are parameters controlling the shift,scale and power respectively. The power activation function is able toyield x and y-antisymmetric activation if c is odd and y-symmetricactivation if c is even. In some implementations, the unit yields anon-rectified linear activation.

In yet other implementations, the convolutional neural network uses asigmoid unit activation function, which is a continuous, saturatingfunction described by the following logistic function:

${\varphi(h)} = \frac{1}{1 + e^{{- \beta}\; h}}$

In the equation above, β=1. The sigmoid unit activation function doesnot yield negative activation and is only antisymmetric with respect tothe y-axis.

Dilated Convolutions

FIG. 13 illustrates dilated convolutions. Dilated convolutions,sometimes called atrous convolutions, which literally means with holes.The French name has its origins in the algorithme a trous, whichcomputes the fast dyadic wavelet transform. In these type ofconvolutional layers, the inputs corresponding to the receptive field ofthe filters are not neighboring points. This is illustrated in FIG. 13.The distance between the inputs is dependent on the dilation factor.

Sub-Sampling Layers

FIG. 14 is one implementation of sub-sampling layers in accordance withone implementation of the technology disclosed. Sub-sampling layersreduce the resolution of the features extracted by the convolutionlayers to make the extracted features or feature maps-robust againstnoise and distortion. In one implementation, sub-sampling layers employtwo types of pooling operations, average pooling and max pooling. Thepooling operations divide the input into non-overlapping two-dimensionalspaces. For average pooling, the average of the four values in theregion is calculated. For max pooling, the maximum value of the fourvalues is selected.

In one implementation, the sub-sampling layers include poolingoperations on a set of neurons in the previous layer by mapping itsoutput to only one of the inputs in max pooling and by mapping itsoutput to the average of the input in average pooling. In max pooling,the output of the pooling neuron is the maximum value that resideswithin the input, as described by:φ_(o)=max(φ₁,φ₂, . . . ,φ_(N))

In the equation above, N is the total number of elements within a neuronset.

In average pooling, the output of the pooling neuron is the averagevalue of the input values that reside with the input neuron set, asdescribed by:

$\varphi_{o} = {\frac{1}{N}{\sum\limits_{n = 1}^{N}\varphi_{n}}}$

In the equation above, N is the total number of elements within inputneuron set.

In FIG. 14, the input is of size 4×4. For 2×2 sub-sampling, a 4×4 imageis divided into four non-overlapping matrices of size 2×2. For averagepooling, the average of the four values is the whole-integer output. Formax pooling, the maximum value of the four values in the 2×2 matrix isthe whole-integer output.

Convolution Examples

FIG. 15 depicts one implementation of a two-layer convolution of theconvolution layers. In FIG. 15, an input of size 2048 dimensions isconvolved. At convolution 1, the input is convolved by a convolutionallayer comprising of two channels of sixteen kernels of size 3×3. Theresulting sixteen feature maps are then rectified by means of the ReLUactivation function at ReLU1 and then pooled in Pool 1 by means ofaverage pooling using a sixteen channel pooling layer with kernels ofsize 3×3. At convolution 2, the output of Pool 1 is then convolved byanother convolutional layer comprising of sixteen channels of thirtykernels with a size of 3×3. This is followed by yet another ReLU2 andaverage pooling in Pool 2 with a kernel size of 2×2. The convolutionlayers use varying number of strides and padding, for example, zero,one, two and three. The resulting feature vector is five hundred andtwelve (512) dimensions, according to one implementation.

In other implementations, the convolutional neural network usesdifferent numbers of convolution layers, sub-sampling layers, non-linearlayers and fully connected layers. In one implementation, theconvolutional neural network is a shallow network with fewer layers andmore neurons per layer, for example, one, two or three fully connectedlayers with hundred (100) to two hundred (200) neurons per layer. Inanother implementation, the convolutional neural network is a deepnetwork with more layers and fewer neurons per layer, for example, five(5), six (6) or eight (8) fully connected layers with thirty (30) tofifty (50) neurons per layer.

Forward Pass

The output of a neuron of row x, column y in the l^(th) convolutionlayer and k^(th) feature map for ƒ number of convolution cores in afeature map is determined by the following equation:

$O_{x,y}^{({l,k})} = {\tan\;{h\left( {{\sum\limits_{t = 0}^{f - 1}{\sum\limits_{r = 0}^{k_{h}}{\sum\limits_{c = 0}^{k_{w}}{W_{({r,c})}^{({k,t})}O_{({{x + r},{x + c}})}^{({{l - 1},t})}}}}} + {Bias}^{({l,k})}} \right)}}$

The output of a neuron of row x, column y in the l^(th) sub-sample layerand k^(th) feature map is determined by the following equation:

$O_{x,y}^{({l,k})} = {\tan\;{h\left( {{W^{(k)}{\sum\limits_{r = 0}^{S_{h}}{\sum\limits_{c = 0}^{S_{w}}O_{({{{x \times S_{h}} + r},{{y \times S_{w}} + c}})}^{({{l - 1},k})}}}} + {Bias}^{({l,k})}} \right)}}$

The output of an i^(th) neuron of the l^(th) output layer is determinedby the following equation:

$O_{({l,i})} = {\tanh\left( {{\sum\limits_{j = 0}^{H}{O_{({{l - 1},j})}W_{({i,j})}^{l}j}} + {Bias}^{({l,i})}} \right)}$Backpropagation

The output deviation of a k^(th) neuron in the output layer isdetermined by the following equation:d(O _(k) ^(o))=y _(k) −t _(k)

The input deviation of a k^(th) neuron in the output layer is determinedby the following equation:d(I _(k) ^(o))=(y _(k) −t _(k))φ′(v _(k))=φ′(v _(k))d(O _(k) ^(o))

The weight and bias variation of a k^(th) neuron in the output layer isdetermined by the following equation:ΔW _(k,x) ^(o))=d(I _(k) ^(o))y _(k,x)ΔBias_(k) ^(o))=d(I _(k) ^(o))

The output bias of a k^(th) neuron in the hidden layer is determined bythe following equation:

${d\left( O_{k}^{H} \right)} = {\sum\limits_{i = 0}^{i < {84}}{{d\left( I_{i}^{o} \right)}W_{i,k}}}$

The input bias of a k^(th) neuron in the hidden layer is determined bythe following equation:d(I _(k) ^(H))=φ′(v _(k))d(O _(k) ^(H))

The weight and bias variation in row x, column y in a m^(th) feature mapof a prior layer receiving input from k neurons in the hidden layer isdetermined by the following equation:ΔW _(m,x,y) ^(H,k))=d(I _(k) ^(H))y _(x,y) ^(m)ΔBias_(k) ^(H))=d(I _(k) ^(H))

The output bias of row x, column y in a m^(th) feature map of sub-samplelayer S is determined by the following equation:

${d\left( O_{x,y}^{S,m} \right)} = {\sum\limits_{k}^{170}{{d\left( I_{m,x,y}^{H} \right)}W_{m,x,y}^{H,k}}}$

The input bias of row x, column y in a m^(th) feature map of sub-samplelayer S is determined by the following equation:d(I _(x,y) ^(S,m))=φ′(v _(k))d(O _(x,y) ^(S,m))

The weight and bias variation in row x, column y in a m^(th) feature mapof sub-sample layer S and convolution layer C is determined by thefollowing equation:

$\left. {{{\Delta W^{S,m}} = {\sum\limits_{x = 0}^{fh}{\sum\limits_{y = 0}^{fw}{{d\left( I_{{\lbrack{x/2}\rbrack},{\lbrack{y/2}\rbrack}}^{S,m} \right)}O_{x,y}^{C,m}}}}}{\Delta Bias}^{S,m}} \right) = {\sum\limits_{x = 0}^{fh}{\sum\limits_{y = 0}^{fw}{d\left( O_{x,y}^{S,m} \right)}}}$

The output bias of row x, column y in a k^(th) feature map ofconvolution layer C is determined by the following equation:d(O _(x,y) ^(C,k))=d(I _([x/)2],[y/2]^(S,k))W ^(k)

The input bias of row x, column y in a k^(th) feature map of convolutionlayer C is determined by the following equation:d(I _(x,y) ^(C,k))=φ′(v _(k))d(O _(x,y) ^(C,k))

The weight and bias variation in row r, column c in an m^(th)convolution core of a k^(th) feature map of l^(th) convolution layer C:

$\left. {{{\Delta W_{r,c}^{k,m}} = {\sum\limits_{x = 0}^{fh}{\sum\limits_{y = 0}^{fw}{{d\left( l_{x,y}^{C,k} \right)}O_{{x + r},{y + c}}^{{l - 1},m}}}}}{\Delta Bias}^{C,k}} \right) = {\sum\limits_{x = 0}^{fh}{\sum\limits_{y = 0}^{fw}{d\left( I_{x,y}^{C,k} \right)}}}$

In one implementation, the nutrition data 426 further comprises amountof processed food servings, amount of natural food, amount of organicfood, amount of genetically modified organism food, amount of netprotein, amount of net carbohydrate, amount of net fat, amount of nettransfat, amount of net saturated fat, amount of net high-densitycholesterol, amount of net low-density cholesterol, amount of netvitamin A, amount of net vitamin B, amount of net vitamin C, amount ofnet vitamin D, amount of net vitamin E, amount of net iron, amount ofnet sodium, amount of net calcium, amount of net magnesium, amount ofnet potassium, and amount of net fiber.

FIG. 5 shows one implementation of the data processing system 502. Thedata processing system 502 processes the avatar data 138, the user data306, and the nutrition data 426 and produces environment interactiondata 512.

FIG. 6 depicts one example of the environment interaction data 512. Theenvironment interaction data 512 includes (i) metadata 602 about thefood offerings and the avatar's response 612 to the food offerings, (ii)time spent 622 by the avatar in different health states, and (iii) theavatar's fitness 632.

In one implementation, the metadata 602 about the food offerings furthercomprises frequency of food presented to the avatar in the digitalvaccine environment, mathematical pattern of food choices presented tothe avatar in the digital vaccine environment, and velocity of foodchoices presented to the avatar in the digital vaccine environment.

In another implementation, the metadata 602 about the avatar's responseto the food offerings further comprises number of interactions theavatar has with healthy food in the digital vaccine environment, numberof interactions the avatar has with pseudo-healthy food in the digitalvaccine environment, number of interactions the avatar has withunhealthy food in the digital vaccine environment, duration of theinteractions, and velocity vector of the interactions.

In one implementation, the time spent 622 by the avatar in differenthealth states further comprises total time spent by the avatar at thedifferent stages of the digital vaccine environment, total time spent bythe avatar in a fit health state, total time spent by the avatar in adanger health state, and total time spent by the avatar in an unhealthyhealth state.

In one implementation, the avatar's fitness 632 further comprisesavatar's movement speed.

FIG. 7 shows one implementation of the modification system 702. Themodification system 702 modifies parameters 712 of the digital vaccineenvironment 100, the avatar 126, and the stages 108 based on theenvironment interaction data 512.

The parameters 712 of the digital vaccine environment 100, the avatar126, and the stages 108 further comprise number of enemies in thedigital vaccine environment, number of robots Non player characters(NPCs) in the digital vaccine environment, strength of the enemy NPCs,type of the enemy NPCs, percentage of enemy enemy NPCs, type of friendlyNPC pets, accuracy of enemy NPCs, velocity of enemy NPCs, virtual foodspawn location, levels and two-dimensional (2D) and three-dimensionalaugment reality and virtual reality assets selection, neurocognitivetraining module selection, nutrition facts module, game level up menu,avatar mesh shape, leaderboard on/off, avatar powerup menu on/off,avatar customization marketplace, game feature reconfiguration setting,level of virtual target selection, level of real-world target selection,and score target.

The data processing system 502 and the modification system 702 can beany type of machine learning systems and can be trained to configure theparameters 712 of the digital vaccine environment 100, the avatar 126,and the stages 108. The training can be supervised, unsupervised, and/orsemi-supervised. Some examples of the machine learning systems that canbe used by the data processing system 502 and the modification system702 include support vector machines, discriminant analysis, naïve bayes,nearest neighbor, decision trees, K-means, hierarchical, gaussianmixture, hidden markov model, eXtreme Gradient Boosted trees, and neuralnetworks. Also, the different types of neural networks that can be usedby the data processing system 502 and the modification system 702 arelisted in FIG. 17.

FIG. 16 illustrates one implementation of a training stage in which thedata processing system 502 and the modification system 702 are trainedon training data to configure the parameters 712 of the digital vaccineenvironment 100, the avatar 126, and the stages 108. The goal oftraining the data processing system 502 and the modification system 702is optimization of the weight parameters in each layer, which graduallycombines simpler features into complex features so that the mostsuitable hierarchical representations can be learned from the trainingdata. A single cycle 1600 of the optimization process is organized asfollows. First, given a training dataset 1602 to the data processingsystem 502 and the modification system 702 under training, the forwardpass sequentially computes the output 1604 in each layer and propagatesthe function signals forward through the network. In the final outputlayer, an objective loss function measures error 1606 between theinferenced outputs 1604 and the ground truth 1608.

To minimize the prediction error, the backward pass 1614 uses the chainrule to backpropagate error signals and compute gradients with respectto all weights throughout the neural network. Finally, the weightparameters are updated using optimization algorithms based on stochasticgradient descent. Whereas batch gradient descent performs parameterupdates for each complete dataset, eXtreme Gradient Boosting can providestochastic approximations by performing the updates for each small setof data examples. In some implementations, the data processing system502 and the modification system 702 is trained on a training data set ofat least a hundred thousand examples of paired ground truth using abackpropagation-based gradient update technique.

FIG. 18 depicts one implementation of a method for artificialintelligence-controlled neuro physiological-behavior state modulation tolower health risk score.

At action 1801, user gains access to software on a computing deviceSmartphone/Tablet/AR/VR head-mounted device.

At action 1802, the digital vaccine environment 100 invokes VR/AR/AImodule and chat-multiplayer networking modules.

At action 1803, the digital vaccine environment 100 presents, to theusers, real world and virtual targets and notification of achievements.

At action 1804, the AI controller of the digital vaccine environment 100selects and presents a precision mapped AR/VR/interactive CG basedneurocognitive modulation-training puzzle module.

At action 1805, the real time and asynchronous multiplayer module of thedigital vaccine environment 100 allows the users to interact withfriends.

At action 1806, the digital vaccine environment 100 invokes evaluationmodule.

At action 1807, the candidate exits AR/VR/interactive CG neurocognitivetraining module.

At action 1808, the digital vaccine environment 100 computes score andrewards earned.

At action 1809, the digital vaccine environment 100 progresses to thenext stage and saves the telemetry and uploads it to a cloud database.

At action 1810, the digital vaccine environment 100 checks whether thedesired neurobehavior state has been achieved.

At action 1811, if the desired neuro physiological-behavior state hasbeen achieved, then the digital vaccine environment 100 invokes rewardsmodule and/or avatar customization marketplace and/or player skillupgrade module.

At action 1812, the digital vaccine environment 100 pushes updates todatabase via a leaderboard module and send notifications via afriend-list module.

At action 1813, the candidate progresses to the next game level andtarget.

At action 1814, if the desired neuro physiological behavior state hasnot been achieved, then the digital vaccine environment 100 invokes anAI-based adaptive tutoring module.

At action 1815, the AI-based adaptive tutoring module then reconfiguresparameters of the digital vaccine environment 100 for this user and herspecific current condition.

At action 1816, the reconfigured parameters of the digital vaccineenvironment 100 are saved into the cloud database.

FIG. 19 depicts one implementation of a method for personalization ofprecision health risk mapping.

User data can be entered into the digital vaccine environment 100 by anyof the entities 1900 listed in FIG. 19.

At action 1904, the entity accepts terms of use and privacy policy.

At action 1905, the teacher/parent/doctor enters email and/or cell phonenumber and/or authentication is done via fingerprint, facialrecognition, voice recognition.

At action 1906, one time passcode is generated to validate email orphone.

At action 1907, the teacher/parent/doctor enters code to validate.

At action 1908, a determination is made whether the child user accountalready exists in the database.

At action 1909, if the child user account already exists in thedatabase, then the child user account is prepopulated with detailsfetched from database.

At action 1910, then the child and guardian confirms/edits details andsubmits for account creation/retrieval.

At action 1911, the initial account validation is complete.

At action 1912, various asset parameters are reconfigured.

At action 1913, the asset parameters are provided as input to the assetreconfiguration AI module 1901 to generate parameters 1903 of thedigital vaccine environment 100. Similarly, the child user information1902 is provided as input to the asset reconfiguration AI module 1901 tofurther generate parameters 1903 of the digital vaccine environment 100.Additionally, the digital vaccine environment 100 is configured withdata from the device 1913.

If the child user account does not exist in the database, then the childname (1914), the child gender (1915), the child date of birth (1917),the child school name (1918), the child grade/class (1919), the childclass section (1920), and the child house or school team (1922) areidentified to and specified in the digital vaccine environment 100.

At action 1916, auto-validation maps the child user characteristics andinitializes the avatar with corresponding gender, in-game choice, chat,and friend/caregiver input, i.e., assets of the digital vaccineenvironment 100.

At action 1921, auto-validation maps the child user characteristics andinitializes the game (the digital vaccine environment 100) withcorresponding game level and the child user characteristics.

Finally, additional child user information such as height (1923), weight(1924), food allergies (1925), chronic diseases health conditions,medications/supplements (1927), cuisine preferences (1928), food grouppreferences (1929), genetic profile/family history (1930), clinicaltests results (1931) are used for auto-validation of the digital vaccineenvironment 100.

At action 1932, the body mass index and risk are compared withclassmates/friends list/team and with family members.

Computer System

FIG. 20 is a simplified block diagram of a computer system 2000 that canbe used to implement the technology disclosed. Computer system 2000includes at least one central processing unit (CPU) 2072 thatcommunicates with a number of peripheral devices via bus subsystem 2055.These peripheral devices can include a storage subsystem 2010 including,for example, memory devices and a file storage subsystem 2036, userinterface input devices 2038, user interface output devices 2076, and anetwork interface subsystem 2074. The input and output devices allowuser interaction with computer system 2000. Network interface subsystem2074 provides an interface to outside networks, including an interfaceto corresponding interface devices in other computer systems.

In one implementation, the data processing system 502 and/or themodification system 702 are communicably linked to the storage subsystem2010 and the user interface input devices 2038.

User interface input devices 2038 can include a keyboard; pointingdevices such as a mouse, trackball, touchpad, or graphics tablet; ascanner; a touch screen incorporated into the display; audio inputdevices such as voice recognition systems and microphones; and othertypes of input devices. In general, use of the term “input device” isintended to include all possible types of devices and ways to inputinformation into computer system 2000.

User interface output devices 2076 can include a display subsystem, aprinter, a fax machine, or non-visual displays such as audio outputdevices. The display subsystem can include an LED display, a cathode raytube (CRT), a flat-panel device such as a liquid crystal display (LCD),a projection device, or some other mechanism for creating a visibleimage. The display subsystem can also provide a non-visual display suchas audio output devices. In general, use of the term “output device” isintended to include all possible types of devices and ways to outputinformation from computer system 2000 to the user or to another machineor computer system.

Storage subsystem 2010 stores programming and data constructs thatprovide the functionality of some or all of the modules and methodsdescribed herein. Subsystem 2078 can be graphics processing units (GPUs)or field-programmable gate arrays (FPGAs).

Memory subsystem 2022 used in the storage subsystem 2010 can include anumber of memories including a main random access memory (RAM) 2032 forstorage of instructions and data during program execution and a readonly memory (ROM) 2034 in which fixed instructions are stored. A filestorage subsystem 2036 can provide persistent storage for program anddata files, and can include a hard disk drive, a floppy disk drive alongwith associated removable media, a CD-ROM drive, an optical drive, orremovable media cartridges. The modules implementing the functionalityof certain implementations can be stored by file storage subsystem 2036in the storage subsystem 2010, or in other machines accessible by theprocessor.

Bus subsystem 2055 provides a mechanism for letting the variouscomponents and subsystems of computer system 2000 communicate with eachother as intended. Although bus subsystem 2055 is shown schematically asa single bus, alternative implementations of the bus subsystem can usemultiple busses.

Computer system 2000 itself can be of varying types including a personalcomputer, a portable computer, a workstation, a computer terminal, anetwork computer, a television, a mainframe, a server farm, awidely-distributed set of loosely networked computers, or any other dataprocessing system or user device. Due to the ever-changing nature ofcomputers and networks, the description of computer system 2000 depictedin FIG. 20 is intended only as a specific example for purposes ofillustrating the preferred embodiments of the present invention. Manyother configurations of computer system 2000 are possible having more orless components than the computer system depicted in FIG. 20.

Clauses

The following clauses are disclosed herein:

1. A digital vaccine system, comprising:

a digital vaccine environment which presents a user-driven avatar with(i) tasks that test the avatar's physical fitness and (ii) foodofferings at various stages, wherein the avatar's appearance isresponsive to the avatar's performance on the tasks and selection of thefood offerings;

an input generation system with a virtual input generation sub-systemand an actual input generation sub-system, wherein

-   -   the virtual input generation sub-system monitors the avatar's        progression through the digital vaccine environment and produces        avatar data, including (i) avatar food preference data, (ii)        avatar calorie data, (iii) avatar insulin data, (iv) avatar        glucose data, (v) avatar A1C data, (vi) avatar ketone        data, (vii) avatar cholesterol (HDL/LDL) data, (viii) avatar        amino acid data, (ix) avatar glycoprotein acetyls data, and (x)        avatar gut microbiome data and    -   the actual input generation sub-system accesses a user        information database and produces user data, including (i) user        food preference data, (ii) user calorie data, (iii) user        glycemic data, (iv) user insulin data, (v) user glucose        data, (vi) user A1C data, (vii) user ketone data, (viii) user        cholesterol (HDL/LDL) data, (ix) user amino acid data, (x) user        glycoprotein acetyls data, and (xi) user microbiome data;

a nutrition data generation system that processes (i) food logs, (ii)user conversation files, (iii) user images, and/or (iv) food images andproduces nutrition data;

a data processing system that processes the avatar data, the user data,and the nutrition data and produces environment interaction data,including (i) metadata about the food offerings and the avatar'sresponse to the food offerings, (ii) time spent by the avatar indifferent health states, and (iii) the avatar's fitness; and

a modification system that modifies parameters of the digital vaccineenvironment, the avatar, and the stages based on the environmentinteraction data.

2. The digital vaccine system of clause 1, wherein

the avatar food preference data further comprises

-   -   time stamped virtual food presented to the avatar in the digital        vaccine environment, and    -   time stamped virtual food selected by the avatar in the digital        vaccine environment;

the avatar calorie data further comprises

-   -   total calorie level of the avatar,    -   calories expanded by the avatar as a result of performing the        tasks, and    -   net calorie level of the avatar;

the avatar insulin data further comprises

-   -   virtual insulin dose counter, and    -   virtual insulin units;

the avatar glucose data further comprises

-   -   virtual net blood glucose;

the avatar A1C data further comprises

-   -   virtual A1C results;

the avatar ketone data further comprises

-   -   virtual ketone level;

the avatar cholesterol data further comprises

-   -   virtual LDL/HDL level;

the avatar amino acid data further comprises

-   -   virtual amino acid level;

the avatar gut microbiome data further comprises

-   -   virtual microbiome level; and

the avatar glycoprotein acetyl data further comprises

-   -   virtual glycoprotein acetyls level;        3. The digital vaccine system of clause 1, wherein

the user food preference data further comprises

-   -   time stamped actual food presented to the user in real world,        and    -   time stamped actual food consumed by the user in the real world;

the user calorie data further comprises

-   -   actual calories consumed by the user in the real world, and    -   calories expanded by the user in the real world;

the user glycemic data further comprises

-   -   glycemic index, and    -   glycemic load;

the user insulin data further comprises

-   -   actual insulin dose counter, and    -   actual insulin units;

the user glucose data further comprises

-   -   actual net blood glucose;

the user A1C data further comprises

-   -   actual A1C results;

the user ketone data further comprises

-   -   actual ketone level;

the user cholesterol data further comprises

-   -   actual LDL/HDL level;

the user amino acid data further comprises

-   -   actual amino acid level;

the user gut microbiome data further comprises

-   -   actual microbiome level; and

the user glycoprotein acetyls data further comprises

-   -   actual glycoprotein acetyls level.        4. The digital vaccine system of clause 1, wherein the nutrition        data generation system is a recurrent neural network that        processes the user conversation files and produces the nutrition        data.        5. The digital vaccine system of clause 1, wherein the nutrition        data generation system is a convolutional neural network that        processes the user images and/or the food images and produces        the nutrition data.        6. The digital vaccine system of clause 1, wherein the nutrition        data further comprises:

amount of processed food servings,

amount of natural food,

amount of organic food,

amount of genetically modified organism food,

amount of net protein,

amount of net carbohydrate,

amount of net fat,

amount of net transfat,

amount of net saturated fat,

amount of net high-density cholesterol,

amount of net low-density cholesterol,

amount of net vitamin A,

amount of net vitamin B,

amount of net vitamin C,

amount of net vitamin D,

amount of net vitamin E,

amount of net iron,

amount of net sodium,

amount of net calcium,

amount of net magnesium,

amount of net potassium, and

amount of net fiber.

7. The digital vaccine system of clause 1, wherein the metadata aboutthe food offerings further comprises:

frequency of food presented to the avatar in the digital vaccineenvironment,

mathematical pattern of food choices presented to the avatar in thedigital vaccine environment, and

velocity of food choices presented to the avatar in the digital vaccineenvironment.

8. The digital vaccine system of clause 1, wherein the metadata aboutthe avatar's response to the food offerings further comprises:

number of interactions the avatar has with healthy food in the digitalvaccine environment,

number of interactions the avatar has with pseudo-healthy food in thedigital vaccine environment,

number of interactions the avatar has with unhealthy food in the digitalvaccine environment,

duration of the interactions, and

velocity vector of the interactions.

9. The digital vaccine system of clause 1, wherein the time spent by theavatar in different health states further comprises:

total time spent by the avatar at the different stages of the digitalvaccine environment,

total time spent by the avatar in a fit health state,

total time spent by the avatar in a danger health state, and

total time spent by the avatar in an unhealthy health state.

10. The digital vaccine system of clause 1, wherein the avatar's fitnessfurther comprises:

avatar's movement speed.

11. The digital vaccine system of clause 1, wherein the parameters ofthe digital vaccine environment, the avatar, and the stages furthercomprise:

number of enemies in the digital vaccine environment,

number of enemy Non player characters NPCs in the digital vaccineenvironment,

strength of the enemy NPCs,

type of the enemy NPCs,

percentage of enemy NPCs,

type of friendly NPCs,

accuracy of enemy NPCs,

accuracy of friendly NPCs

velocity of NPCs,

virtual food spawn location,

levels and two-dimensional (2D) and three-dimensional augment realityand virtual reality assets selection,

neurocognitive training module selection,

nutrition facts module,

game level up menu,

avatar mesh shape,

leaderboard on/off,

avatar powerup menu on/off,

avatar customization marketplace,

game feature reconfiguration setting,

level of virtual target selection,

level of real-world target selection, and

score target.

12. A computer-implemented method of providing a digital vaccine system,including:

presenting a user-driven avatar with (i) tasks that test the avatar'sphysical fitness and (ii) food offerings at various stages, wherein theavatar's appearance is responsive to the avatar's performance on thetasks and selection of the food offerings;

monitoring the avatar's progression through the digital vaccineenvironment and producing avatar data, including (i) avatar foodpreference data, (ii) avatar calorie data, (iii) avatar insulin data,(iv) avatar glucose data, (v) avatar A1C data, and (vi) avatar ketonedata, (vii) avatar cholesterol (HDL/LDL) data, (viii) avatar amino aciddata, (ix) avatar glycoprotein acetyls data, and (x) avatar gutmicrobiome data and

accessing a user information database and producing user data, including(i) user food preference data, (ii) user calorie data, (iii) userglycemic data, (iv) user insulin data, (v) user glucose data, (vi) userA1C data, and (vii) user ketone data, (viii) user cholesterol (HDL/LDL)data, (ix) user amino acid data, (x) user glycoprotein acetyls data, and(xi) user microbiome data;

processing (i) food logs, (ii) user conversation files, (iii) userimages, and/or (iv) food images and producing nutrition data;

processing the avatar data, the user data, and the nutrition data andproducing environment interaction data, including (i) metadata about thefood offerings and the avatar's response to the food offerings, (ii)time spent by the avatar in different health states, and (iii) theavatar's fitness; and

modifying parameters of the digital vaccine environment, the avatar, andthe stages based on the environment interaction data.

13. The computer-implemented method of clause 12, wherein

the avatar food preference data further comprises

-   -   time stamped virtual food presented to the avatar in the digital        vaccine environment, and    -   time stamped virtual food selected by the avatar in the digital        vaccine environment; the avatar calorie data further comprises    -   total calorie level of the avatar,    -   calories expanded by the avatar as a result of performing the        tasks, and    -   net calorie level of the avatar;

the avatar insulin data further comprises

-   -   virtual insulin dose counter, and    -   virtual insulin units;

the avatar glucose data further comprises

-   -   virtual net blood glucose;

the avatar A1C data further comprises

-   -   virtual A1C results;

the avatar ketone data further comprises

-   -   virtual ketone level;

the avatar cholesterol data further comprises

-   -   virtual LDL/HDL level;

the avatar amino acid data further comprises

-   -   virtual amino acid level;

the avatar gut microbiome data further comprises

-   -   virtual microbiome level; and

the avatar glycoprotein acetyl data further comprises

-   -   virtual glycoprotein acetyls level.        14. The computer-implemented method of clause 12, wherein

the user food preference data further comprises

-   -   time stamped actual food presented to the user in real world,        and    -   time stamped actual food consumed by the user in the real world;

the user calorie data further comprises

-   -   actual calories consumed by the user in the real world,    -   calories expanded by the user in the real world;

the user glycemic data further comprises

-   -   glycemic index, and    -   glycemic load;

the user insulin data further comprises

-   -   actual insulin dose counter, and    -   actual insulin units;

the user glucose data further comprises

-   -   actual net blood glucose;    -   the user A1C data further comprises    -   actual A1C results;

the user ketone data further comprises

-   -   actual ketone level;

the user cholesterol data further comprises

-   -   actual LDL/HDL level;

the user amino acid data further comprises

-   -   actual amino acid level;

the user gut microbiome data further comprises

-   -   actual microbiome level; and

the user glycoprotein acetyls data further comprises

-   -   actual glycoprotein acetyls level.        15. The computer-implemented method of clause 12, wherein the        time spent by the avatar in different health states further        comprises:

total time spent by the avatar at the different stages of the digitalvaccine environment,

total time spent by the avatar in a fit health state,

total time spent by the avatar in a danger health state, and

total time spent by the avatar in an unhealthy health state.

16. The computer-implemented method of clause 12, wherein the avatar'sfitness further comprises:

avatar's movement speed.

17. The computer-implemented method of clause 12, wherein the parametersof the digital vaccine environment, the avatar, and the stages furthercomprise:

number of NPCs in the digital vaccine environment,

number of enemy NPCs in the digital vaccine environment,

strength of the enemy NPCs,

type of the enemy NPCs,

percentage of enemy NPCs,

type of friendly NPCs,

accuracy of enemy NPCs,

velocity of enemy NPCs,

virtual food spawn location,

levels and two-dimensional (2D) and three-dimensional augment realityand virtual reality assets selection,

neurocognitive training module selection,

nutrition facts module,

game level up menu,

avatar mesh shape,

leaderboard on/off,

avatar powerup menu on/off,

avatar customization marketplace,

game feature reconfiguration setting,

level of virtual target selection,

level of real-world target selection, and

score target.

18. A non-transitory computer readable storage medium impressed withcomputer program instructions to provide a digital vaccine system, theinstructions, when executed on a processor, implement a methodcomprising:

presenting a user-driven avatar with (i) tasks that test the avatar'sphysical fitness and (ii) food offerings at various stages, wherein theavatar's appearance is responsive to the avatar's performance on thetasks and selection of the food offerings;

monitoring the avatar's progression through the digital vaccineenvironment and producing avatar data, including (i) avatar foodpreference data, (ii) avatar calorie data, (iii) avatar insulin data,(iv) avatar glucose data, (v) avatar A1C data, and (vi) avatar ketonedata, (vii) avatar cholesterol (HDL/LDL) data, (viii) avatar amino aciddata, (ix) avatar glycoprotein acetyls data, and (x) avatar gutmicrobiome data and

accessing a user information database and producing user data, including(i) user food preference data, (ii) user calorie data, (iii) userglycemic data, (iv) user insulin data, (v) user glucose data, (vi) userA1C data, and (vii) user ketone data (viii) user cholesterol (HDL/LDL)data, (ix) user amino acid data, (x) user glycoprotein acetyls data, and(xi) user microbiome data;

processing (i) food logs, (ii) user conversation files, (iii) userimages, and/or (iv) food images and producing nutrition data;

processing the avatar data, the user data, and the nutrition data andproducing environment interaction data, including (i) metadata about thefood offerings and the avatar's response to the food offerings, (ii)time spent by the avatar in different health states, and (iii) theavatar's fitness; and

modifying parameters of the digital vaccine environment, the avatar, andthe stages based on the environment interaction data.

19. The non-transitory computer readable storage medium of clause 18,wherein the avatar's fitness further comprises:

avatar's movement speed.

20. The non-transitory computer readable storage medium of clause 18,wherein the parameters of the digital vaccine environment, the avatar,and the stages further comprise:

number of enemies in the digital vaccine environment,

number of enemy NPCs in the digital vaccine environment,

strength of the enemy NPCs,

type of the enemy NPCs,

percentage of enemy NPCs,

type of friendly NPCs,

accuracy of enemy NPCs,

velocity of enemy NPCs,

virtual food spawn location,

levels and two-dimensional (2D) and three-dimensional augment realityand virtual reality assets selection,

neurocognitive training module selection,

nutrition facts module,

game level up menu,

avatar mesh shape,

leaderboard on/off,

avatar powerup menu on/off,

avatar customization marketplace,

game feature reconfiguration setting,

level of virtual target selection,

level of real-world target selection, and

score target.

What is claimed is:
 1. A digital vaccine system, comprising: a digitalvaccine environment, running on one or more hardware processors, whichpresents a user-driven avatar with (i) tasks that test the avatar'sphysical fitness and (ii) food offerings at various stages, wherein theavatar performs the tasks and selects the food offerings, and whereinthe avatars appearance is responsive to the avatar's performance on thetasks and selection of the food offerings; an input generation systemwith a virtual input generation sub-system and an actual inputgeneration sub-system, wherein the virtual input generation sub-systemmonitors the avatar's progression through the digital vaccineenvironment and produces avatar data, including (i) avatar foodpreference data, (ii) avatar calorie data, (iii) avatar insulin data,(iv) avatar glucose data, (v) avatar A1C data, (vi) avatar ketone data,(vii) avatar cholesterol (HDL/LDL) data, (viii) avatar amino acid data,(ix) avatar glycoprotein acetyls data, and (x) avatar gut microbiomedata and the actual input generation sub-system accesses a userinformation database and produces user data, including (i) user foodpreference data, (ii) user calorie data, (iii) user glycemic data, (iv)user insulin data, (v) user glucose data, (vi) user A1C data, (vii) userketone data, (viii) user cholesterol (HDL/LDL) data, (ix) user aminoacid data, (x) user glycoprotein acetyls data, and (xi) user microbiomedata; a nutrition data generation system that processes (i) food logs,(ii) user conversation files, (iii) user images, and/or (iv) food imagesand produces nutrition data; a data processing system that processes theavatar data, the user data, and the nutrition data and producesenvironment interaction data, including (i) metadata about the foodofferings and the avatars response to the food offerings, (ii) timespent by the avatar in different health states, and (iii) the avatar'sfitness; and a modification system that modifies parameters of thedigital vaccine environment, the avatar, and the stages based on theenvironment interaction data.
 2. The digital vaccine system of claim 1,wherein the avatar food preference data further comprises time stampedvirtual food presented to the avatar in the digital vaccine environment,and time stamped virtual food selected by the avatar in the digitalvaccine environment; the avatar calorie data further comprises totalcalorie level of the avatar, calories expanded by the avatar as a resultof performing the tasks, and net calorie level of the avatar; the avatarinsulin data further comprises virtual insulin dose counter, and virtualinsulin units; the avatar glucose data further comprises virtual netblood glucose; the avatar A1C data further comprises virtual A1Cresults; the avatar ketone data further comprises virtual ketone level;the avatar cholesterol data further comprises virtual LDL/HDL level; theavatar amino acid data further comprises virtual amino acid level; theavatar gut microbiome data further comprises virtual microbiome level;and the avatar glycoprotein acetyl data further comprises virtualglycoprotein acetyls level.
 3. The digital vaccine system of claim 1,wherein the user food preference data further comprises time stampedactual food presented to the user in real world, and time stamped actualfood consumed by the user in the real world; the user calorie datafurther comprises actual calories consumed by the user in the realworld, and calories expanded by the user in the real world; the userglycemic data further comprises glycemic index, and glycemic load; theuser insulin data further comprises actual insulin dose counter, andactual insulin units; the user glucose data further comprises actual netblood glucose; the user A1C data further comprises actual Al C results;the user ketone data further comprises actual ketone level; the usercholesterol data further comprises actual LDL/HDL level; the user aminoacid data further comprises actual amino acid level; the user gutmicrobiome data further comprises actual microbiome level; and the userglycoprotein acetyls data further comprises actual glycoprotein acetylslevel.
 4. The digital vaccine system of claim 1, wherein the nutritiondata generation system is a recurrent neural network that processes theuser conversation files and produces the nutrition data.
 5. The digitalvaccine system of claim 1, wherein the nutrition data generation systemis a convolutional neural network that processes the user images and/orthe food images and produces the nutrition data.
 6. The digital vaccinesystem of claim 1, wherein the nutrition data further comprises: amountof processed food servings, amount of natural food, amount of organicfood, amount of genetically modified organism food, amount of netprotein, amount of net carbohydrate, amount of net fat, amount of nettransfat, amount of net saturated fat, amount of net high-densitycholesterol, amount of net low-density cholesterol, amount of netvitamin A, amount of net vitamin B, amount of net vitamin C, amount ofnet vitamin D, amount of net vitamin E, amount of net iron, amount ofnet sodium, amount of net calcium, amount of net magnesium, amount ofnet potassium, and amount of net fiber.
 7. The digital vaccine system ofclaim 1, wherein the metadata about the food offerings furthercomprises: frequency of food presented to the avatar in the digitalvaccine environment, mathematical pattern of food choices presented tothe avatar in the digital vaccine environment, and velocity of foodchoices presented to the avatar in the digital vaccine environment. 8.The digital vaccine system of claim 1, wherein the metadata about theavatar's response to the food offerings further comprises: number ofinteractions the avatar has with healthy food in the digital vaccineenvironment, number of interactions the avatar has with pseudo-healthyfood in the digital vaccine environment, number of interactions theavatar has with unhealthy food in the digital vaccine environment,duration of the interactions, and velocity vector of the interactions.9. The digital vaccine system of claim 1, wherein the time spent by theavatar in different health states further comprises: total time spent bythe avatar at the different stages of the digital vaccine environment,total time spent by the avatar in a fit health state, total time spentby the avatar in a danger health state, and total time spent by theavatar in an unhealthy health state.
 10. The digital vaccine system ofclaim 1, wherein the avatar's fitness further comprises: avatar'smovement speed.
 11. The digital vaccine system of claim 1, wherein theparameters of the digital vaccine environment, the avatar, and thestages further comprise: number of enemies in the digital vaccineenvironment, number of enemy Non player characters NPCs in the digitalvaccine environment, strength of the enemy NPCs, type of the enemy NPCs,percentage of enemy NPCs, type of friendly NPCs, accuracy of enemy NPCs,accuracy of friendly NPCs velocity of NPCs, virtual food spawn location,levels and two-dimensional (2D) and three-dimensional augment realityand virtual reality assets selection, neurocognitive training moduleselection, nutrition facts module, game level up menu, avatar meshshape, leaderboard on/off, avatar powerup menu on/off, avatarcustomization marketplace, game feature reconfiguration setting, levelof virtual target selection, level of real-world target selection, andscore target.
 12. A computer-implemented method of providing a digitalvaccine system, including: presenting a user-driven avatar with (i)tasks that test the avatar's physical fitness and (ii) food offerings atvarious stages, wherein the avatar performs the tasks and selects thefood offerings, and wherein the avatars appearance is responsive to theavatar's performance on the tasks and selection of the food offerings;monitoring the avatar's progression through the digital vaccineenvironment and producing avatar data, including (i) avatar foodpreference data, (ii) avatar calorie data, (iii) avatar insulin data,(iv) avatar glucose data, (v) avatar Al C data, and (vi) avatar ketonedata, (vii) avatar cholesterol (HDL/LDL) data, (viii) avatar amino aciddata, (ix) avatar glycoprotein acetyls data, and (x) avatar gutmicrobiome data and accessing a user information database and producinguser data, including (i) user food preference data, (ii) user caloriedata, (iii) user glycemic data, (iv) user insulin data, (v) user glucosedata, (vi) user A1C data, and (vii) user ketone data, (viii) usercholesterol (HDL/LDL) data, (ix) user amino acid data, (x) userglycoprotein acetyls data, and (xi) user microbiome data; processing (i)food logs, (ii) user conversation files, (iii) user images, and/or (iv)food images and producing nutrition data; processing the avatar data,the user data, and the nutrition data and producing environmentinteraction data, including (i) metadata about the food offerings andthe avatar's response to the food offerings, (ii) time spent by theavatar in different health states, and (iii) the avatar's fitness; andmodifying parameters of the digital vaccine environment, the avatar, andthe stages based on the environment interaction data.
 13. Thecomputer-implemented method of claim 12, wherein the avatar foodpreference data further comprises time stamped virtual food presented tothe avatar in the digital vaccine environment, and time stamped virtualfood selected by the avatar in the digital vaccine environment; theavatar calorie data further comprises total calorie level of the avatar,calories expanded by the avatar as a result of performing the tasks, andnet calorie level of the avatar; the avatar insulin data furthercomprises virtual insulin dose counter, and virtual insulin units; theavatar glucose data further comprises virtual net blood glucose; theavatar A1C data further comprises virtual A1C results; the avatar ketonedata further comprises virtual ketone level; the avatar cholesterol datafurther comprises virtual LDL/HDL level; the avatar amino acid datafurther comprises virtual amino acid level; the avatar gut microbiomedata further comprises virtual microbiome level; and the avatarglycoprotein acetyl data further comprises virtual glycoprotein acetylslevel.
 14. The computer-implemented method of claim 12, wherein the userfood preference data further comprises time stamped actual foodpresented to the user in real world, and time stamped actual foodconsumed by the user in the real world; the user calorie data furthercomprises actual calories consumed by the user in the real world,calories expanded by the user in the real world; the user glycemic datafurther comprises glycemic index, and glycemic load; the user insulindata further comprises actual insulin dose counter, and actual insulinunits; the user glucose data further comprises actual net blood glucose;the user A1C data further comprises actual Al C results; the user ketonedata further comprises actual ketone level; the user cholesterol datafurther comprises actual LDL/HDL level; the user amino acid data furthercomprises actual amino acid level; the user gut microbiome data furthercomprises actual microbiome level; and the user glycoprotein acetylsdata further comprises actual glycoprotein acetyls level.
 15. Thecomputer-implemented method of claim 12, wherein the time spent by theavatar in different health states further comprises: total time spent bythe avatar at the different stages of the digital vaccine environment,total time spent by the avatar in a fit health state, total time spentby the avatar in a danger health state, and total time spent by theavatar in an unhealthy health state.
 16. The computer-implemented methodof claim 12, wherein the avatar's fitness further comprises: avatar'smovement speed.
 17. The computer-implemented method of claim 12, whereinthe parameters of the digital vaccine environment, the avatar, and thestages further comprise: number of NPCs in the digital vaccineenvironment, number of enemy NPCs in the digital vaccine environment,strength of the enemy NPCs, type of the enemy NPCs, percentage of enemyNPCs, type of friendly NPCs, accuracy of enemy NPCs, velocity of enemyNPCs, virtual food spawn location, levels and two-dimensional (2D) andthree-dimensional augment reality and virtual reality assets selection,neurocognitive training module selection, nutrition facts module, gamelevel up menu, avatar mesh shape, leaderboard on/off, avatar powerupmenu on/off, avatar customization marketplace, game featurereconfiguration setting, level of virtual target selection, level ofreal-world target selection, and score target.
 18. A non-transitorycomputer readable storage medium impressed with computer programinstructions to provide a digital vaccine system, the instructions, whenexecuted on a processor, implement a method comprising: presenting auser-driven avatar with (i) tasks that test the avatar's physicalfitness and (ii) food offerings at various stages, wherein the avatarperforms the tasks and selects the food offerings, and wherein theavatars appearance is responsive to the avatar's performance on thetasks and selection of the food offerings; monitoring the avatar'sprogression through the digital vaccine environment and producing avatardata, including (i) avatar food preference data, (ii) avatar caloriedata, (iii) avatar insulin data, (iv) avatar glucose data, (v) avatar AlC data, and (vi) avatar ketone data, (vii) avatar cholesterol (HDL/LDL)data, (viii) avatar amino acid data, (ix) avatar glycoprotein acetylsdata, and (x) avatar gut microbiome data and accessing a userinformation database and producing user data, including (i) user foodpreference data, (ii) user calorie data, (iii) user glycemic data, (iv)user insulin data, (v) user glucose data, (vi) user A1C data, and (vii)user ketone data (viii) user cholesterol (HDL/LLDL) data, (ix) useramino acid data, (x) user glycoprotein acetyls data, and (xi) usermicrobiome data; processing (i) food logs, (ii) user conversation files,(iii) user images, and/or (iv) food images and producing nutrition data;processing the avatar data, the user data, and the nutrition data andproducing environment interaction data, including (i) metadata about thefood offerings and the avatar's response to the food offerings, (ii)time spent by the avatar in different health states, and (iii) theavatar's fitness; and modifying parameters of the digital vaccineenvironment, the avatar, and the stages based on the environmentinteraction data.
 19. The non-transitory computer readable storagemedium of claim 18, wherein the avatar's fitness further comprises:avatar's movement speed.
 20. The non-transitory computer readablestorage medium of claim 18, wherein the parameters of the digitalvaccine environment, the avatar, and the stages further comprise: numberof enemies in the digital vaccine environment, number of enemy NPCs inthe digital vaccine environment, strength of the enemy NPCs, type of theenemy NPCs, percentage of enemy NPCs, type of friendly NPCs, accuracy ofenemy NPCs, velocity of enemy NPCs, virtual food spawn location, levelsand two-dimensional (2D) and three-dimensional augment reality andvirtual reality assets selection, neurocognitive training moduleselection, nutrition facts module, game level up menu, avatar meshshape, leaderboard on/off, avatar powerup menu on/off, avatarcustomization marketplace, game feature reconfiguration setting, levelof virtual target selection, level of real-world target selection, andscore target.